Papers with routing quality
Exploration-Driven Reinforcement Learning for Expert Routing Improvement in Mixture-of-Experts Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | MoE-based LLMs are not explicitly supervised to select suitable experts. |
| Approach: | They propose Exploration-Driven Reinforcement Learning (ERL) which explicitly optimizes the router by exploration of alternative routing paths. |
| Outcome: | The proposed method improves summarization (SAMSum, XSUM, question answering, and language modeling), and raises routing quality, delivering 8.9 higher MRR than baselines over 100 perturbed routing paths. |